Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(arm64-support): add arm64 to goreleaser config #81

Merged
merged 3 commits into from
Jan 3, 2022

Conversation

samandmoore
Copy link
Contributor

@samandmoore samandmoore commented Dec 18, 2021

hi there! 👋🏻

I just got my Tidbyt in the mail (yay!) and started playing around. I had to build from source because I'm using an M1 Mac.

I have a bunch of experience with golang, so I decided to see if I could get things updated to fix this issue #39

This PR is an attempt at that 😄

Notes
This adds arm64 to the list of supported go architectures in the
goreleaser configuration.

It also updates the fetch-artifacts.sh script to copy the linux and
darwin arm64 artifacts. I'm not entirely sure how this fits into the
buildkite CI process, but it seems like it might need to be updated.

I'm able to run the goreleaser command locally and see it generate all
the proper files.

One interesting and important note about building the render module
while running on macos, i did need to set up the CPATH and LIBRARY_PATH
as described here #39 (comment)

fixes #39 (I hope)

this adds arm64 to the list of supported go architectures in the
goreleaser configuration.

it also updates the fetch-artifacts.sh script to copy the linux and
darwin arm64 artifacts. i'm not entirely sure how this fits into the
buildkite CI process, but it seems like it might need to be updated.

i'm able to run the goreleaser command locally and see it generate all
the proper files.

one interesting and important note about building the render module
while running on macos, i did need to set up the CPATH and LIBRARY_PATH
as described here tidbyt#39 (comment)
@rohansingh
Copy link
Member

Hey @samandmoore, thanks for the PR! I'm investigating how this fits into our CI as well. I think it's going to take some work on our part here.

Currently we're building the final binaries natively, on a Linux amd64 agent and a macOS amd64 agent. So I think we'll have to spin up a macOS arm64 build agent.

@samandmoore
Copy link
Contributor Author

Ah gotcha. That's interesting. Ideally you'd be able to use golang cross compilation. Do you know what is preventing that? Is it the linking to the webp library?

@rohansingh
Copy link
Member

Yup, exactly. Since it's a native dependency we can't use cross-compilation (without setting up an entire cross toolchain, anyway).

@betterengineering
Copy link
Contributor

Update, I'm able to cross-compile Linux amd64 -> Linux arm64, though I've only validated that the produced binary does not execute on an Intel CPU 😄 :

# Install cross compilation tools
sudo apt install -y crossbuild-essential-arm64

# Add arm64 based libewebp-dev package
sudo dpkg --add-architecture arm64
sudo apt update
sudo apt install libwebp-dev:arm64

Then we can run something like this to cross compile with CGO:

CC=aarch64-linux-gnu-gcc CGO_ENABLED=1 GOOS=linux GOARCH=arm64 go build -o pixlet tidbyt.dev/pixlet
./pixlet 
zsh: exec format error: ./pixlet
  • CC=aarch64-linux-gnu-gcc tells system what gcc compiler to use
  • CGO_ENABLED=1 says even though we're cross-compiling, don't disable CGO since go does that normally
  • The ARCH and OS are what you would expect

I think we can follow a similar pattern for MacOS and I'm validating that now

@betterengineering
Copy link
Contributor

Update, I found an arm64 based linux machine to test on and confirmed it can render the hello_world.star!

@betterengineering
Copy link
Contributor

Good news! I had a chance to look at this again after the holidays and was able to get this to cross compile for M1 as well. I downloaded libwebp-1.2.2-rc1-mac-arm64.tar.gz from here since it was the only version with MacOS ARM support: https://storage.googleapis.com/downloads.webmproject.org/releases/webp/index.html

Then I was able to reference it to build against:

mspicer@americano:~/code/src/tidbyt.dev/pixlet|main ⇒  CGO_CFLAGS="-I/Users/mspicer/Downloads/libwebp-1.2.2-rc1-mac-arm64/include" CGO_LDFLAGS="-L/Users/mspicer/Downloads/libwebp-1.2.2-rc1-mac-arm64/lib" CGO_ENABLED=1 GOOS=darwin GOARCH=arm64 go build -o pixlet tidbyt.dev/pixlet

Again, was only able to confirm it is not able to run on my machine since I don't currently have an M1 to test against:

mspicer@americano:~/code/src/tidbyt.dev/pixlet|main ⇒  ./pixlet
zsh: bad CPU type in executable: ./pixlet
mspicer@americano:~/code/src/tidbyt.dev/pixlet|main ⇒  file pixlet
pixlet: Mach-O 64-bit executable arm64

@betterengineering
Copy link
Contributor

Alright! I'm ready to land this. I might still need to make some tweaks here. I'm working on getting everything into GitHub actions so that it's all open source in #104

@betterengineering betterengineering merged commit 84046d8 into tidbyt:main Jan 3, 2022
@samandmoore
Copy link
Contributor Author

Awesome! Excited to see this coming together!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Webp not working on M1 Mac (Apple Silicon)
3 participants